A stochastic quasigradient algorithm with variable metric
نویسندگان
چکیده
منابع مشابه
A Self-Correcting Variable-Metric Algorithm for Stochastic Optimization
An algorithm for stochastic (convex or nonconvex) optimization is presented. The algorithm is variable-metric in the sense that, in each iteration, the step is computed through the product of a symmetric positive definite scaling matrix and a stochastic (mini-batch) gradient of the objective function, where the sequence of scaling matrices is updated dynamically by the algorithm. A key feature ...
متن کاملVariable Metric Stochastic Approximation Theory
We provide a variable metric stochastic approximation theory. In doing so, we provide a convergence theory for a large class of online variable metric methods including the recently introduced online versions of the BFGS algorithm and its limited-memory LBFGS variant. We also discuss the implications of our results in the areas of eliciting properties of distributions using prediction markets a...
متن کاملVariable Metric Stochastic Approximation Theory
We provide a variable metric stochastic approximation theory. In doing so, we provide a convergence theory for a large class of online variable metric methods including the recently introduced online versions of the BFGS algorithm and its limited-memory LBFGS variant. We also discuss the implications of our results for learning from expert advice.
متن کاملA block coordinate variable metric forward-backward algorithm
A number of recent works have emphasized the prominent role played by the KurdykaLojasiewicz inequality for proving the convergence of iterative algorithms solving possibly nonsmooth/nonconvex optimization problems. In this work, we consider the minimization of an objective function satisfying this property, which is a sum of a non necessarily convex differentiable function and a non necessaril...
متن کاملNovel Stochastic Gradient Adaptive Algorithm with Variable Length
The goal of this paper is to present a novel variable length LMS (Least Mean Square) algorithm, in which the length of the adaptive filter is always a power of two and it is modified using an error estimate. Unlike former variable length stochastic gradient adaptive techniques, the proposed algorithm works in non-stationary situations. The implementation of the adaptive filter is described and ...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Annals of Operations Research
سال: 1992
ISSN: 0254-5330,1572-9338
DOI: 10.1007/bf02060944